Displaying representative images in a visual mapping system

ABSTRACT

Embodiments provide systems and methods for generating a street map that includes a position identifier that identifies a location on the street map. The method and system may also generate and display a plurality of images representative of the location of the position identifier. A user may interact with a position identifier or one of several scroll icons to view images of other locations on the street map and/or to obtain driving directions between two locations.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/811,564, filed Nov. 13, 2017, which is a continuation of U.S. patentapplication Ser. No. 15/282,849, filed Sep. 30, 2016, now U.S. Pat. No.9,818,173, which is a continuation of U.S. patent application Ser. No.14/873,088, filed Oct. 1, 2015, now U.S. Pat. No. 9,710,886, which is acontinuation of U.S. patent application Ser. No. 14/102,128, filed Dec.10, 2013, now U.S. Pat. No. 9,182,895, which is a continuation of Ser.No. 12/551,405, filed Aug. 31, 2009, now U.S. Pat. No. 8,606,493, whichis a continuation of Ser. No. 12/536,407, filed Aug. 5, 2009, now U.S.Pat. No. 8,543,323, which is a divisional of Ser. No. 11/396,369, filedMar. 31, 2006, now U.S. Pat. No. 7,587,276, which is acontinuation-in-part of U.S. patent application Ser. No. 11/045,007,filed Jan. 26, 2005, now U.S. Pat. No. 7,359,797, which is acontinuation-in-part of U.S. patent application Ser. No. 10/809,049,filed Mar. 24, 2004, now U.S. Pat. No. 7,155,336, each of which ishereby incorporated herein by reference.

BACKGROUND

Various online directories and map services, such as online Yellow Pagesand MapQuest®, are available for users to locate and find informationabout businesses and services and directions to those businesses.Typically, when searching an online business directory, a user isprompted to enter selection criteria, such as the general geographiclocation (e.g., state and city), and the type or category of business(e.g., restaurant) that the user is looking for, or perhaps the name ofthe business itself, if known. In response, listings for each of thebusinesses that meet the user's selection criteria are displayed. Eachlisting typically identifies the name, address, and phone number of thelisted business. Further, each listing may be associated with icons thatthe user may click to obtain further information about the business,such as a map of an area around the business and driving directions tothe business.

Current online directories, however, do not provide listings that can bedisplayed with an image of the physical location of the businessfeatured in the listing. While some aerial or rooftop images areavailable online in correlation with addresses, such aerial or rooftopviews are of limited value because they do not show, for example,business buildings in the manner actually perceived by customers (i.e.,at the ground or street level). Additionally, online maps do not provideimages of physical surroundings associated with points on a map. Moreparticularly, what is lacking is an automated system and method forcollecting and displaying images of objects at geographic locations,such as business storefronts and street views of structures, in onlinedirectories and maps.

BRIEF SUMMARY

The present invention provides methods and systems whereby online maps,such as street maps, may be displayed along with images of building andother objects at physical locations represented by those maps. Thus, auser accessing an online map prepared using embodiments of the presentinvention can visually appreciate the businesses, buildings, streets,and also perhaps areas surrounding a location.

In accordance with one aspect, a computer-readable medium havinginstructions stored thereon that directs a computing system to display amap and images representative of a location on that map is provided. Aposition identifier that includes a pair of scroll icons may bedisplayed on the map to identify a particular location. For thatlocation, a plurality of images may be displayed to illustrate thephysical surroundings (e.g., buildings) at the location identified bythe position identifier. Those images may be displayed in a dualfilmstrip-type view, one on each side of a graphical representation of astreet. The orientation of the dual filmstrip-type view may be such thatit illustrates the images on either side of a street. Alternatively, thedual filmstrip-type view may provide a montage, panoramic, or any othertype view, of objects surrounding the location identified by theposition identifier. A set of scroll icons may be positioned at eitherend of the filmstrip-type views. Those scroll icons may be directionallycorrelated with the scroll icons displayed on the position identifierand interaction with any of the scroll icons will have an effect on thecorrelated scroll icons.

In accordance with another aspect, a method for providing directionsbetween a first location and a second location is provided. The methoddetermines a route between the first location and the second locationand displays direction details for the route. The displayed directiondetails include links to an image representative of the locationsincluded or otherwise referenced in the driving directions. A map mayalso be displayed that graphically emphasis the driving directions.

In accordance with another aspect of the present invention, a computersystem having a computer-readable medium including a computer-executableprogram therein for performing a method for providing informationregarding a location is provided. The method performed by the computersystem selects a location and displays, on a graphical user interface, aposition identifier on a street map at the selected location. Aplurality of images associated with the selected location are alsodisplayed on the graphical user interface. The plurality of images maybe displayed in a graphical order. For example, they may be arrangedsuch that they are represented on either side of a representation of astreet. Alternatively, the images may be images only along one side ofthe street.

Accordingly, an online map system of the present invention permits theuser to visually appreciate not only street maps and images of variouslocations within the street maps, but also images surrounding thoselocations. A user may also interact with the scroll icons to obtain newimages thereby virtually moving up or down the street.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of thisinvention will become more readily appreciated as the same become betterunderstood by reference to the following detailed description, whentaken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a pictorial flow diagram of one exemplary method forcollecting address-correlated images of objects at geographic locationsfor publication in an online directory according to one embodiment ofthe present invention;

FIG. 2 is a functional block diagram of an exemplary system that may beused to collect images of objects at geographic locations;

FIG. 3 is a schematic diagram illustrating distance offsetting between acamera and an object being imaged by the camera;

FIG. 4 is a flow diagram of one exemplary method for preparing images ofobjects collected at geographic locations for publication in an onlinedirectory according to one embodiment of the present invention;

FIG. 5 is a block diagram showing a street block having three addresses,each of which has a defined boundary for which images may be correlated,in accordance with an embodiment of the present invention;

FIG. 6 is a block diagram showing an environment and arrangement ofcertain computing components for implementing an online directoryaccording to one embodiment of the present invention;

FIG. 7 is a pictorial diagram of a graphical user interface displayingan example of a listings page returned to a client system when a userrequests to view the listings matching a search criteria, generatedusing embodiments of the present invention;

FIG. 8 is a pictorial diagram of a graphical user interface displayingan example of a listing detail page returned to a client system when auser requested detailed information for a particular listing, generatedusing embodiments of the present invention;

FIGS. 9A-9E depict examples of graphical user interfaces in the form ofWeb pages received from a directory server and displayed on a clientsystem, illustrating sample operations of an online directory inaccordance with the present invention;

FIG. 10 is a flow diagram of a search results routine utilized toprovide listing information in response to a user-provided searchrequest, in accordance with an embodiment of the present invention;

FIG. 11 is a flow diagram of a details information routine forpresenting a detailed information view of a selected address, inaccordance with an embodiment of the present invention;

FIG. 12 is a flow diagram of a voting subroutine for designating arepresentative image for a geographic location in response to userselections, in accordance with an embodiment of the present invention;

FIG. 13 is a flow diagram of an image correlation correction routine foradjusting the address coordinates for the geographic location of aparticular address and re-correlating images to a particular address,according to an embodiment of the present invention;

FIG. 14 is a flow diagram of an address boundary definition subroutinefor defining address boundaries, in accordance with an embodiment of thepresent invention;

FIGS. 15-23 illustrate sample screen displays of map pages, that can bedisplayed on a client system in response to user requests, searchqueries, and interactions with the map page, in accordance withembodiments of the present invention;

FIG. 24 is a flow diagram of a map directions routine for displayingdirections between two locations along with representative imagesbetween those two locations, in accordance with an embodiment of thepresent invention; and

FIG. 25 is a block diagram showing an environment and arrangement ofcertain computing components for implementing an online mapping systemaccording to one embodiment of the present invention.

DETAILED DESCRIPTION

The present invention includes systems and methods for collecting anddisplaying images of objects (including, without limitation, buildings,structures, and store fronts) obtained at different geographiclocations. The images may be displayed in association with correspondinglistings in online directories. The following detailed descriptionprovides several exemplary implementations of the invention. Althoughspecific system configurations and flow diagrams are illustrated, itshould be understood that the examples provided are not exhaustive anddo not limit the invention to the precise forms disclosed. Personshaving ordinary skill in the field of digital data processing willrecognize that the computer components and arrangements described hereinmay be interchangeable with other components and arrangements and thatthe process steps described herein may be interchangeable with othersteps or their combinations, and still achieve the benefits andadvantages of the present invention.

It should also be noted that while the following description is providedin the context of maps with graphical images of locations included in amap, and an online business directory (e.g., Yellow Pages) at which eachbusiness listing is associated with an image of its geographic location,the present invention can be applied to create and publish onlineresidential directories (e.g., White Pages) in which each individuallisting is associated with an image of its geographic location (e.g., animage of a house). Therefore, the term “Yellow Pages” as used herein mayrefer to other types of directories as well, including White Pages.

FIG. 1 illustrates an overall flow diagram of a method for collectingimages of objects at geographic locations for publication in an onlinedirectory, at which each listing is associated with an image taken by acamera at the address in the listing. A system for collecting images(e.g., system 16 in FIG. 2) is mounted on a vehicle 9. The vehicle 9 maybe an automobile, as illustrated, but may be of any other type apparentto one skilled in the art, such as a bicycle, bus, or airborne vehiclesuch as a manned or unmanned airplane. The image collection system, inthis embodiment, includes at least one camera 10 and a GPS receiver 12.Optionally, a range finder 14, a rate of travel monitor 25, and a rateof turn monitor 27 may also be included in the system and mounted on thevehicle 9.

FIG. 2 illustrates an embodiment of a system 16 suitable for collectingimages of objects at geographic locations and correlating the imageswith street addresses for the geographic locations. The system 16includes the camera 10, the GPS receiver 12, and perhaps the rangefinder 14, rate of travel monitor 25, and rate of turn monitor 27, eachincluding or having access to a synchronized clock 11, 13, 15, 21, and29, respectively. The clocks 11, 13, 15, 21, and 29 may be separateclocks that are synchronized with each other or, alternatively, a singleclock to which the camera 10, the GPS receiver 12, the range finder 14,the rate of travel monitor 25, and the rate of turn monitor 27 arecoupled for time reference. A synchronized clock may thus mean one ormore clocks from which each of multiple devices can receive synchronizedtime information. The camera 10, the GPS receiver 12, the range finder14, and rate of travel monitor 25 and rate of turn monitor 27, ifprovided, are coupled to a computing device 18, such as a laptopcomputer. In some applications where the data storage capacity of thecamera 10, the GPS receiver 12, the range finder 14, the rate of travelmonitor 25, and the rate of turn monitor 27 is sufficiently large, thecomputing device 18 need not be coupled to the camera 10, the GPSreceiver 12, the range finder 14, the rate of travel monitor 25, and therate of turn monitor 27 during image collection (e.g., while mounted onthe vehicle 9).

The computing device 18 includes a processor 19 in communication with aninput/output interface 20 and a memory 22. The input/output interface 20enables the computing device 18 to communicate with various input andoutput devices. Input devices may include the camera 10, the GPSreceiver 12, the range finder 14, the rate of travel monitor 25, and therate of turn monitor 27, as illustrated, plus any other computingelements that provide input signals to the computing device 18, such asa keyboard, mouse, external memory, disk drive, etc.

Output devices 23 may include typical output devices, such as a computerdisplay, printer, facsimile machine, copy machine, etc. The processor 19is configured to operate in accordance with computer programinstructions stored in a memory, such as the memory 22. Programinstructions may also be embodied in a hardware format, such as one ormore programmed digital signal processors. The memory 22 may also beconfigured to store various data (e.g., image data, geographic locationdata, and speed data) collected and processed, as will be more fullydescribed below. In some applications, the data obtained by the camera10, the GPS receiver 12, and the range finder 14 (if provided) are sentdirectly to a hard drive (forming part of the memory 22) of thecomputing device 18 for storage, instead of being first stored in videotapes or removable disks. The memory 22 may include, for example, alookup table database 24 that stores address data in correlation withgeographic locations (e.g., longitude and latitude coordinates).Suitable lookup table databases are routinely used for the purpose ofcreating maps and are commercially available. Those having ordinaryskill in the art of computers will recognize that a wide selection ofcommercially available components and arrangements can be used toconstruct a system, such as the system 16 illustrated in FIG. 2. Inparticular, the computing device 18 may include additional conventionalcomponents, such as a network interface, which are not illustratedherein for the purpose of clarity.

Referring back to FIG. 1, once the camera 10, the GPS receiver 12, andperhaps the range finder 14, rate of travel monitor 25 and/or rate ofturn monitor 27 are mounted on the vehicle 9, an operator drives thevehicle while automatically, or perhaps manually (with the aid ofanother operator), capturing a series of images of objects (e.g.,building or store fronts) on each side of the street using the camera10. In some applications, for example, when the street width issufficiently narrow or the street is relatively quiet, two cameras 10may be mounted on the vehicle facing opposite directions so as tosimultaneously take two series of images covering both sides of thestreet. In other applications, three or more cameras 10 may be mountedon the vehicle. For example, two or more cameras 10 may be mounted onthe same side of the vehicle (for example, two on one side and two onthe other side) to collectively obtain a stereoscopic image of eachobject being imaged. Any suitable auto-focus camera(s) 10 may be used,as will be apparent to one skilled in the art, though preferably thecamera 10 would be a digital still camera or a digital video camera. Ifa digital still camera is used, the series of images taken will be stillimages. If a digital video camera is used, the series of images takenwill be image frames that collectively form a video image. In any case,the camera 10 preferably time-stamps each image, i.e., records the timeeach image (or each image frame) is taken, in reference to asynchronized clock 11. Time-stamping is a standard feature ofcommercially available cameras.

As the camera 10 captures images of objects at geographic locations(e.g., businesses), the GPS receiver 12 records the geographic locations(e.g., longitude and latitude coordinates) while preferablytime-stamping the recorded location data. The camera 10 and the GPSreceiver 12 may be positioned or programmed relative to each other(e.g., by offsetting the distance therebetween) such that the geographiclocation determined by the GPS receiver 12 can be treated as that of thecamera 10 itself. The rate of travel monitor 25, if used, records therate of travel of the vehicle 9 as data is being collected whilepreferably time-stamping such information. As discussed below, the rateof travel information may be used to assist in filtering images or indetermining the geographic coordinates of an image. Similarly, the rateof turn monitor 27, if used, records the degree of any turns made by thevehicle 9 as data is being collected while preferably time-stamping suchinformation. For example, the rate of turn monitor 27 may be a gyroscopemounted on the vehicle 9, and the resistance incurred by the gyroscopeduring a turn may be used to calculate the degree of the turn made.Calculating and recording the degree of a turn may be used inconjunction with the rate of travel information recorded by the rate oftravel monitor 25 to determine the geographic coordinates of an image ifGPS data is not available.

The range finder 14, if used, records the distance to an object beingviewed by the range finder 14 while preferably time-stamping suchdistance information. The range finder 14 may also be positioned orprogrammed relative to the camera 10 and the GPS receiver 12 such thatthe distance information determined by the range finder 14 can betreated as that of the camera 10. Any suitable GPS receiver and rangefinder may be used, as will be apparent to one skilled in the art. Forexample, some high-precision GPS receivers suitable for use in thepresent invention are available from Trimble Navigation Ltd. ofSunnyvale, Calif. (trimble.com). A line of laser range finders suitablefor use in the present invention is available from Measurement DevicesLtd. (MDL) of Scotland, U.K. (mdl.co.uk).

The range finder 14 is useful in offsetting the distance between the GPSreceiver 12 (geographically coinciding with the camera 10 and the rangefinder 14) and the object being imaged by the camera 10, so as to moreaccurately determine the precise geographic location of the object beingimaged. Specifically, referring to FIG. 3, when a camera 10 mounted on avehicle is moved from position 10′ to position 10″ with the movement ofthe vehicle, the distance from the camera to the object being imagedchanges from distance A (to building 26) to distance B (to building 28).If the distance from the camera 10 to the objects being imaged by thecamera 10 is substantially constant, a standard offset (e.g., distanceA) may be used to calculate the longitude/latitude coordinates (i.e.,geographic location) of the objects being imaged. If, on the other hand,this distance varies substantially, or if it is preferred to preciselycalculate the longitude/latitude coordinates of the objects beingimaged, the range finder 14 may be used to continuously measure thisdistance. The direction of the field of view of the camera 10 may bedetermined by a compass (not shown) or based on the direction of themovement of the camera 10 carried by the vehicle (e.g., from position10′ to position 10″ in FIG. 3) as determined by the GPS receiver 12. Anangle of the camera 10 relative to the moving direction of the vehicleis known and may be used to further determine the direction of thecamera 10 relative to the objects being imaged.

Sometimes the vehicle 9 must travel on inclines. Since buildings arebuilt perpendicular to the earth, with the camera 10 being mounted levelto the vehicle 9, the images captured by the camera 10 appear tilted. Toovercome this problem, a gyroscope can be used with the camera 10mounted on a gimbal. A suitable gyroscope-based mounting system (or astabilizer) can be obtained from Kenyon Laboratories LLC of Essex, Conn.(ken-lab.com). The gyroscope-based mounting system, in an embodiment ofthe present invention may use the same gyroscope used to gather data forthe rate of turn monitor 27. Alternatively, separate gyroscopes may beused. Another approach for solving the problem of images appearingtilted is to use a digital inclinometer to track an angle of the camera10 relative to a level, and then rotate the images in software duringpost-capture processing to compensate for the measured tilt-angles.

To collect images and geographic locations of numerous businesses, manyvehicles may be used, each equipped with a camera and a GPS receiver. Inorder to efficiently collect images and geographic location data ofbusinesses in remote or isolated areas, certain utility vehicles, suchas U.S. Postal Service delivery vehicles or taxicabs, may be requestedto carry a data collection system of the present invention.

Referring back to FIG. 1, the camera 10 therefore produces a set oftime-stamped image data 30, the GPS receiver 12 produces a set oftime-stamped geographic location data 32 (e.g., longitude and latitudecoordinates), the range finder 14 produces a set of time-stampeddistance information data 34, the rate of travel monitor 25 produces aset of time-stamped rate of travel information data 35, and the rate ofturn monitor 27 produces a set of time-stamped rate of turn informationdata 37. These sets of data may be downloaded to the computing device 18(see FIG. 2) and associated with each other based on their respectivetime-stamps so as to associate each image with a geographic location ofthe object featured in the image, as indicated at block 36 in FIG. 1.The data download to the computing device 18 may occur continuously(e.g., at the same time as the data is collected by the camera 10, theGPS receiver 12, the range finder 14, and the rate of travel monitor 25)or in batches (e.g., after a certain volume of data is collected by thecamera 10, the GPS receiver 12, the range finder 14, the rate of travelmonitor 25, and the rate of turn monitor 27).

Alternative methods of associating image data with geographic locationsare encompassed by the invention. For example, instead of relying onsynchronization of the clocks 11, 13, 15, and 21 of the respectivedevices, the devices may be coupled to each other so that, for example,image taking of the camera 10 instantaneously triggers the GPS receiver12 and the range finder 14 to record the geographic location and thedistance information at that location, and the data are recordedtogether as a unit. Alternatively, geographic location recording of theGPS receiver 12 (or another controller triggered by the geographiclocation recording of the GPS receiver 12) triggers the camera 10 totake an image. For example, the longitude/latitude coordinates ofselected businesses whose images are to be taken (available from thelookup table database 24) may be input to the GPS receiver 12 coupled tothe camera 10. Each time the GPS receiver 12 determines it is located infront of a selected business, the GPS receiver 12 records the geographiclocation and triggers the camera 10 to record an image of the business.In this way, only those images capturing listed business addresses willbe taken. The foregoing description is not exhaustive, and other methodsfor associating images with geographic locations may be used, as will beapparent to one skilled in the art.

While components of the system 16, such as the camera 10, the GPSreceiver 12, the range finder 14, the rate of travel monitor 25, and thecomputing device 18, may be individually configured and assembled asdescribed above, alternative systems may employ various commerciallyavailable geographic information systems (GIS) that are typically usedfor surveying and mapping. Typically, a geographic information system isa combination of elements (including both hardware and software)designed to collect, manipulate, store, and display geographic data. Forexample, Red Hen Systems, Inc., of Fort Collins, Colo.(redhensystems.com), offers add-on GPS receivers sold under thetrademark of VMS 200™ and VMS Mobile™ that can be used with anoff-the-shelf digital video camera to embed GPS data on the videorecording. Similarly, Measurement Devices Ltd. (MDL) of Scotland, U.K.(mdl.co.uk), offers an add-on GPS module sold under the trademark ofVideoAce® that can be combined with any video camera to collect videostill images and movies that are referenced with GPS data. Using thesedevices, one can readily collect a series of images that are referencedwith their geographic locations (longitude and latitude coordinates).Note that use of these systems combining the GPS receiver and the cameraeliminates the need to synchronize clocks in various components (thecamera 10, the GPS receiver 12, etc.).

According to one aspect of the present invention, only a subset of theentire set of obtained images (video image frames, or still images) maybe selected and used so as to minimize both gaps and overlaps. Forexample, when a video camera is set to take 30 frames per second, onlyframes every X feet (depending on the rate of travel of the videocamera) are selected to minimize both frame gaps and frame overlaps.Alternatively, or in addition thereto, every X frame may be selected tofurther minimize both frame gaps and frame overlaps. Still further, inanother embodiment, a subset of images may be obtained by comparingadjacent images and retaining those images that are sufficientlydifferent, based upon a predetermined threshold, from one another.

Sometimes, signals from the GPS satellites may become too weak to permitthe GPS receiver 12 to calculate its geographic location. For example,the GPS receiver 12 may not function well when it is placed without aclear view of a wide sky (e.g., in cities with tall buildings, undertrees with a thick canopy, etc.). Also, there may be limitations on thelogging rate of the GPS receiver 12 (e.g., most GPS receivers currentlypermit only one geographic location recordation per second). Inaccordance with the present invention, interpolation of the geographiclocation information, rate of travel information and rate of turninformation, if gathered, is used to overcome the GPS signal loss issuesand/or the limitations on the GPS logging rate. Specifically, using anytwo geographic locations recorded by the GPS with time-stamps (“loggedlocations”), the rate of travel (if unknown), and also the direction oftravel can be calculated. Then, the geographic location (e.g., longitudeand latitude coordinates) of any point between the two logged locationscan be calculated using interpolation based on the rate of travel andthe direction of travel. Additionally, if rate of turn information wasgathered, any turns between the two geographic locations recorded by theGPS may be determined. Additionally, the logged locations can beoverlaid onto existing maps showing streets, which define feasibleregions (routes) of image capture along which the vehicle 9 carrying thecamera 10 could have passed. In other words, the streets indicatefeasible (valid) paths along which the geographic locations calculatedbased on interpolation can fall. For example, if a map shows a building,any interpolated geographic location (between two logged locations) thatfalls “inside” the building is invalid as showing the possible locationof the camera 10 because the vehicle 9 carrying the camera 10 could nothave driven “through” the building. Therefore, by overlaying the loggedlocations to existing maps of streets, which provide feasible regions toverify the validity of any geographic location calculated based oninterpolation, one can reduce or eliminate interpolation errors.Accordingly, using the interpolation method, perhaps as assisted by themap-overlaying method, a geographic location of any image can becalculated even when such a geographic location cannot be directlyobtained using the GPS receiver 12.

In some cases, it is preferable to use a GPS receiver having arelatively higher refresh rate so as to reduce the need forinterpolation. For example, while most commercially available GPSreceivers currently permit only one geographic location recordation persecond, a GPS receiver and its associated software may be configured toobtain more than one geographic location recordation per second.

Next, as indicated in block 38 of FIG. 1, the computing device 18correlates each image (already associated with a geographic location)with an address by referring to a lookup table database 24 that storesaddresses in correlation with their longitude and latitude coordinates.Alternatively, a suitable algorithm for deriving an address based on ageographic location (e.g., longitude and latitude coordinates) may beused, such as the algorithm supported by “Geocoding Server” availablefrom MapQuest™. Such an algorithm uses a series of logics to estimate anaddress based on longitude/latitude coordinates, such as a blocknumbering system, a position within a sequence of addresses on a block(e.g., 1002 is the second of 10 addresses on a block so it is likely tobe at 20% of the block distance from the corner), and the odd/evennumbering system to indicate a particular side of each street. Based onthe estimated location of an address derived from an algorithm, imageshaving coordinates near the estimated location may be correlated withthat address. In addition, as discussed in more detail below withrespect to FIGS. 5 and 14, a boundary surrounding the estimated locationof each address may be defined and images within that boundary may becorrelated with that address. Further details regarding the process 39(comprising steps 36 and 38 of FIG. 1) for manipulating data collectedby the camera 10, the GPS receiver 12, and the range finder 14 areprovided in reference to FIGS. 4 and 5.

After the series of images are correlated with addresses in block 38,the address-correlated images may be used in publishing an onlinedirectory 40 at which each listing 72 (FIG. 1) can be displayed togetherwith an actual image 74 of the address in the listing. In particular,one of the correlated images may be selected as a representative image,or best image, for the address. For example, if there are several imagescorrelated with an address, the correlated image with geographiccoordinates that are the closest to the geographic coordinates of theaddress may be selected as the representative image for that address.Alternatively, as discussed in more detail below, images may be voted onby users and an image with the most votes may be selected as therepresentative image for an address. The publication and operation of anonline directory will be more fully described below.

Referring now to FIG. 4, a method of manipulating the data collected bythe camera 10, the GPS receiver 12, and the range finder 14, forproducing address-correlated image data is described. In block 50,images of objects (e.g., business structures) taken by the camera 10 arereceived wherein each image is time-stamped. In block 51, a series ofgeographic locations (longitude and latitude coordinates) that are alsotime-stamped are received from the GPS receiver 12. In block 52, if therange finder 14 is used, a series of time-stamped distance informationobtained by the range finder 14 is received. In block 53, the imagedata, the longitude and latitude coordinate data, and the distanceinformation (if obtained), are all associated with each other based ontheir respective time-stamps, to thereby associate each image with theparticular latitude/longitude coordinates of the object featured in theimage. As described above, only a subset of the entire set of images maybe selected and used to minimize gaps and overlaps. Also as describedabove, when the geographic locations of certain images cannot beobtained directly from the GPS receiver 12, those locations can becalculated based on interpolation. Thereafter, in block 55, each imageis correlated with an address. Multiple images may be correlated to thesame address. Likewise, the same image may be correlated to multipleaddresses. Correlation may be accomplished in any suitable manner, suchas by assigning a corresponding address to each image, or by assigning acommon identification number to each image and its address.

In block 56, each image is stored in correlation with its address. Notethat, if a specialty mapping/surveying device that produces image dataalready embedded with GPS data is used, as described above, then blocks50-53 may be replaced with a single block 54 at which such GPSdata-referenced image data is received. As before, at this time, only asubset of the entire set of images (or image frames) captured by thecamera 10 may be selectively received to minimize gaps and overlaps, andalso, any geographic locations not directly obtainable from the GPSreceiver 12 may be calculated based on interpolation. Thereafter, asbefore, the images are correlated in block 55 with addresses inreference to the lookup table 24, and the address-correlated images arestored at block 56.

Regardless of whether the system utilizes the process described withrespect to blocks 50-53, or utilizes images already embedded with GPSdata, as illustrated by block 54, the images may be correlated toaddresses based on a boundary defined for each address. In oneembodiment, address coordinates may be estimated using an algorithm,such as the algorithm supported by “Geocoding Server” available fromMapQuest™. Alternatively, the total number of addresses on one side of ablock may be determined and boundaries defined for each address thatequally divide the block between addresses. In another embodiment, theblock may be geographically divided based on an estimated size of thestructures at each address. The estimated size of the structure and theestimated location of the address may be determined based on the numberof addresses on a block in conjunction with the types of businesses,and/or number of businesses, occupying those structures. Business typesmay be obtained from a listings database 67 (FIG. 6). For each addressboundary, regardless of what technique is used to define the boundary,all images associated with longitude/latitude coordinates within thatboundary may be correlated to that address.

Referring to FIG. 5, if one side of a block 501 has three addresses 501,503, 505 and it is known that the structure at address 501 contains agrocery store (i.e., it is likely a large structure), the structure ataddress 503 contains a shoe repair store (i.e., likely a smallstructure), and the structure at address 503 contains a conveniencestore (i.e., it is likely a medium size structure), the geographic areaof the block may be divided based upon an estimated size of thestructures at each address. Thus, for each address a boundary may bedefined that encompasses the estimated size of the structure located atthat particular address. For example, a large boundary 507 may bedefined for the grocery store located at address 501, a small boundary509 may be defined for the shoe repair store 503, and a medium sizeboundary may be defined for the convenience store 505.

A boundary defined for an address may be any shape and size, and theexample of a rectangle illustrated in FIG. 5 is utilized for explanationpurposes only. As will be appreciated, any other shape such as a circle,square, polygon, etc., may be used to define a geographic areasurrounding an address. Additionally, boundaries may be defined suchthat they overlap one another, thereby allowing images falling withinboth boundaries to be assigned to multiple addresses. For example,boundaries 509 and 511 are defined such that they overlap 513. Allimages having associated longitude/latitude coordinates within theoverlap 513 may be assigned to both address 503 and address 505.Overlapping boundary definitions ensures that all images obtained for aparticular block are correlated to at least one address. Alternatively,boundaries may be defined for addresses such that there is no overlap.Defining boundaries for addresses increases the range with which imagesmay be identified for correlation, thereby potentially increasing thenumber of images correlated to an address.

Returning to FIG. 4, in addition to correlating images for addresses,for each address, one image may be selected and stored as arepresentative image for that address, as illustrated by block 57. Asdiscussed in more detail below, a representative image for an object maybe displayed to a user as part of an address listing. For example,referring briefly to FIG. 7, for each address included in the addresslisting 707, a representative image, such as image 727, is provided. Therepresentative image may be, for example, an image of the store front ofthe business. The representative image may be selected in a variety ofways. For example, the image with longitude/latitude coordinates thatare nearest the longitude/latitude coordinates of the address, obtainedfrom the lookup table 24, may be identified as the representative imagefor that address. As will be described in more detail below, therepresentative image, also referred to herein as the “best image,” maybe adjusted based on user input. For example, users may vote as to whichof the images correlated with an address should be the representativeimage for that address. Those votes may be compiled and the image with amajority of the votes may be selected and stored as the representativeimage. Alternatively, a user, such as a business owner, may upload animage that is to be used as the representative image for an address.Allowing users to provide input (e.g., votes or images) for therepresentative image for an address and adjusting the representativeimage based on that input allows the system to improve its accuracy asto the images displayed in an online directory. In yet another example,the representative image may be selected by determining which of thecorrelated images is the best quality (e.g., clearest) image.Alternatively, the representative image may be the correlated image thatwas taken most recently in time. In another embodiment, a representativeimage may be selected based on the current season. For example, if it isthe Christmas season, a representative image taken of the businessduring Christmas time may be selected. Alternatively, if it is summer,an image of the business taken during the summer may be selected as therepresentative image. As will be appreciated by one of skill in therelevant art, there are numerous ways for selecting a representativeimage and those mentioned above are only provided as examples. Anyrepresentative image selection technique may be used with embodiments ofthe present invention. Adjusting the representative image and othercorrelated images based on user input will be described in more detailwith respect to FIGS. 8-13.

Address-correlated images, as described above, may be published in anonline directory, or displayed as additional mapping information, asdescribed in more detail below. FIG. 6 illustrates one environment forimplementing an online Yellow Pages that includes a Yellow Pages system60 and one or more client systems 61 (only one is shown), such aspersonal computers and PDAs that individual users can use to communicatewith the Yellow Pages system 60. In the environment shown in FIG. 6, theclient systems 61 communicate with the Yellow Pages system 60 via wiredor wireless connections to one or more computer networks, such as theInternet 62. Protocols and components for communicating via the Internetare well known to those of ordinary skill in the art of computer networkcommunications. The client system 61 requests and retrieves informationfrom the Yellow Pages system 60 via a Web browser application 69 thatprovides the user with a graphical user interface to the Internet, as isalso well known to those of ordinary skill in the art.

The Yellow Pages system 60 depicted in FIG. 6 operates in a distributedcomputing environment comprising several computer systems that areinterconnected via communication links, e.g., using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that the system 60 could equally operate ina computer system having fewer or greater number of components than areillustrated in FIG. 6. Thus, the description of the Yellow Pages system60 in FIG. 6 should be taken as exemplary and not limiting to the scopeof the invention. In particular, the Yellow Pages system 60 may compriseplural subsystems, each for performing a specific function in responseto a request from a client system.

The Yellow Pages system 60 as illustrated includes both a Yellow Pagesserver 63 having an operating system 64 and a search engine 65, and adatabase server 66 having a listings database 67 and an images database68. The Yellow Pages server 63, in this example, is a Web page serverthat supports an online Yellow Pages Web site on which each listing(e.g., business) can be displayed together with an image correlated withthe listing. The Yellow Pages server 63 communicates text and graphicsorganized as a Web page using, for example, hypertext transferprotocols, in response to requests and search queries received from theclient systems 61. The Yellow Pages server 63 also conducts searches ofthe databases 67 and 68 in the database server 66. The administrationand overall operation of the Yellow Pages server 63 is controlled by itsoperating system 64, while its search function is performed by thesearch engine 65 in cooperation with the database server 66.

The database server 66 maintains the listings database 67 and the imagesdatabase 68 that the Yellow Pages server 63 uses to respond to userrequests and search queries. The listings database 67 containsinformation generally found in conventional Yellow Pages in a searchableform, i.e., numerous listings (business names together with theiraddresses, phone numbers, business type, etc.) searchable by theirbusiness names, addresses, phone numbers, business types, etc. Thelistings database 67 may also contain information such as the businesshours, a brief description or profile, customer reviews, etc., of eachbusiness listed. The images database 68 contains digitized images of thebusinesses in correlation with the business addresses stored in thelistings database 67. The database server 66 is configured to receivesearch instructions from the search engine 65 and return search resultsfrom the listings database 67 and the images database 68. Those ofordinary skill in the art will recognize that the Yellow Pages server 63and the database server 66 will each typically employ a memory and mainprocessor in which program instructions are stored and executed foroperation of the servers.

The Yellow Pages system 60 enables a user to search and view onlineYellow Pages in which each listing (e.g., business) is displayed with arepresentative image of an object (e.g., business building) taken at thegeographic location of the business. Additionally, the online YellowPages may also include a street map identifying the location of eachlisting. FIGS. 7, 8, and 9A-9E illustrate sample graphical userinterfaces, provided in the form of Web pages that can be displayed onthe client system 61 in response to user requests and search queries.

FIG. 7 illustrates a listings page 700 returned to a client system 61when the user requests to view the listings for “pizza” 705 surroundinga particular location, in this example, “New York, N.Y. 10025” 701. Thelocation for which the search is to be performed may be provided by auser submitting limitations (e.g., city, state, or zip code) or based onknown user preferences, such as the user's home or work address.Additionally, a user may specify the distance 703 from the location 701for which the search is to be performed.

The listings 707 matching the search criteria are presented in thelistings page 700. Each listing includes the name of the business, suchas “Sal's & Carmine Pizza” 713, the address of the business, such as“2671 Broadway, New York, N.Y. 10025” 723, the phone number of thebusiness, such as “(212) 663-7651” 725, a representative image 727 ofthe business, and the distance 729 of the business from the userspecified location. The display of a business listing is described inmore detail with respect to FIGS. 9A-9E. The representative image may beone of the images previously correlated with the address in the listing(i.e., an image of the actual structure containing the business), theimages either being previously taken or provided (e.g., uploaded) by auser. Additionally, the representative image may be a user-specificrepresentative image (i.e., an image identified by the user performingthe search as the preferred representative image). Alternatively, theimage may be any image associated with the business, selected randomlyor based on some other criteria.

The listings 707 may be organized based on any number of criteria. Forexample, the listings 707 may be organized by distance from the userspecified location or alphabetically by business name. As will beappreciated by one of skill in the relevant art, the listings may beorganized in any manner. The listings page 700 may also be configured topresent all results matching the search criteria or a subset of theresults with a link to additional pages containing additional matchingresults.

The listings page 700 may also include a street map 709 identifying theuser specified location 711, and the location of each the businessesidentified in the listing 707. For example, “Sal's & Carmine Pizza” 713included in the listing 707 as the first item is identified in thestreet map 709 with a corresponding graphical icon, or identifier 715.The identifier may by the number of the business as included in thelisting 707, an image of the structure in which the business is located,the trademark of the business, or any other identifier of the business.

The identifier 715, as well as each of the business names andcorresponding images in the listing 707, may be interactive. Forexample, if a user positions a pointing device, such as a mouse pointer,over the business name or image included in the listing 707 thecorresponding identifier included in the map 709 may be emphasized.Likewise, if a user positions a pointing device over an identifier, thecorresponding business and image in the listing 707 may be emphasized.An identifier or listing may be emphasized by highlighting, changing thecolor, changing the size, changing the shape, or performing any otherform of visual alteration on the identifier, image, or business name tothereby distinguish it from other material included on the listings page700. For example, if a user positions a mouse pointer (not shown) overthe business name “Colosseo Romano Restaurant Pizzeria Inc.” 717, orover the corresponding image 719, the identifier 721 on the map 709 maybe highlighted, thereby visually distinguishing the identifier 721 fromthe street map 709 and the other identifiers included on the street map709.

In addition to the business names, corresponding images, and identifiersbeing interactive in that they may be emphasized, selection of any oneof the business name, corresponding image, or identifier may result in alisting detail page 800 (FIG. 8) providing additional information aboutthe selected business. A listing detail page 800 may also be presentedin response to a user providing search criteria that result in only asingle match. For example, if a user searches for “World Famous Ray'sPizza” in New York, N.Y., the listing detail page 800 may be presentedinstead of the listings page 700 (FIG. 7) as the results only returnedone match.

For a selected business, a variety of information may be included in thelisting detail page 800. For example, information such as the businessname 801, business address 803, business telephone number 805, therepresentative image for the business 807, and a street map 809identifying the location of the business may be displayed. Initially,the representative image may be the same representative image presentedin the listings page 700 for the address. However, as described below,through interaction with the listing detail page 800, a user may alterthe image that is presented as the representative image. As an example,for the representative image 807, the listing detail page 800 mayinclude a “zoom in” icon 810, a “walk to the left” icon 813, and a “walkto the right” icon 815, any of which may be selected by a user torequest a specific operation through the browser application 69 of theclient system 61. Selection of the “walk to the left” icon 813 or “walkto the right” icon 815 may result in a different image, such as thegeographically adjacent correlated image, being presented as therepresentative image.

A secondary group of correlated images 811 may also be included in thelisting detail page 800 identifying other images that have beencorrelated with the address of the business 801 identified on thelisting detail page 800. In one example, the secondary group of imagesmay be presented in a filmstrip-type view by ordering and displaying theimages in a geographic sequence. For example, the filmstrip-type view ofsecondary images 811 may include six images 817, 819, 821, 823, 825, and827. However, any number of images may be displayed. Additionally, inanother embodiment, the images may be ordered and displayed based on thetime at which the images were taken. Additionally, the images presentedas the secondary images may be selected based on the season. Forexample, if the current season is fall, the secondary images presentedmay be images correlated with that address which were taken in the fall.

Each of the secondary group of images 817-827 may be interactive in thata user may position a pointing device, such as a mouse pointer, over oneof the images and that image will temporarily replace the representativeimage 807. Additionally, the secondary group of images 811 may alsoinclude a “walk to the left” icon 832 and a “walk to the right” icon834, allowing a user to view additional correlated images to the left orright of the business identified in the listing detail page 800. Forexample, if a user selects “walk to the right” icon 834, the nextgeographic image correlated with the business would be displayed in thesecondary set of images 811. Still further, the secondary group ofimages 811 may include a “scroll to the left” icon 831 and a “scroll tothe right” icon 833, allowing a user to view the next set of correlatedimages to the left or right of the currently displayed set of images.For example, if a user selects “scroll to the left” icon 831, six newimages are displayed as the correlated images 811. The six new imagesdisplayed are the six images that are geographically to the left of thecurrent set of displayed correlated images.

Still further, in an embodiment of the present invention, a user mayspecify, or vote, which image of the secondary group of images 811 theyperceive as being the appropriate representative image for the businessidentified in the listing detail page 800. For example, a user mayspecify that image 823 is the appropriate representative image for thebusiness “World Famous Ray's Pizza” 801 because the image includes acentered view of the structure containing “Ray's Pizza.” Such selection,or voting, may be accomplished by a user selecting (or clicking) thebest image box 835 under the image 823. In response to a selection by auser of a representative image, the system 60 stores that image as theuser's own personal choice of the representative image for that businessat that address. If the user, at a later time, uses the system 60 andobtains information for “World Famous Ray's Pizza,” either in a listingspage 700 or listing detail page 800, the user selected image will bepresented as the representative image. Additionally, the user's vote forthat particular image as the best image is saved with respect to thatimage and the corresponding business address and used to determine whatimage should be used as a default representative image for the business.A default representative image is provided as the representative imagefor a business to users who have not selected a representative image forthat business. As discussed below in more detail, with respect to FIG.12, votes may be tallied from multiple user selections as to what imageshould be the representative image, and the image with the majority ofvotes may be used as the default representative image for a particularbusiness.

In an embodiment of the present invention, a user may also provideinformation and images with respect to a business address via thelisting detail page 800. For example, a user may upload his or her ownimages that may be correlated with a particular business or address. Inparticular, a business owner may upload an image of the store front thatmay be associated with the owner's business and provided as the defaultrepresentative image. In addition to images being provided andcorrelated with a particular business, other information, such as menus,hours of operation, employee information, and other types ofbusiness-related information may also be provided (e.g., uploaded) andassociated with a particular business. Such other business-relatedinformation may be included on the listing detail page 800.

Also included in the listing detail page 800 may be a link 839 tosimilar types of businesses. For example, the system 60 may determinefrom the listings database 67 that “World Famous Ray's Pizza” 801 is ofa business type “pizza” and thereby provide a link 839 to otherbusinesses of the type “pizza.”

In addition, links 837 to businesses nearby (e.g., on the same street)the selected business may also be provided in the listing detail page800. The links 837 to nearby businesses may include the business name,business address, telephone number, and optionally, the representativeimage for the business. The links 837 may be divided up to illustratelinks 841 to businesses on the same side of the street as the currentbusiness and links 843 to businesses on the other side of the street.

Alternatively, or in addition thereto, the listing detail page 800 mayinclude a link, such as “what's on the other side of the street” (notshown), that if selected will provide a user with information and imagesfor the businesses on the other side of the street. Additionally, such alink may present a full image of the other side of the street. Such animage helps to give additional context to the location of the businessfor which information is provided in listing detail page 800.Alternatively, a link such as “See other business on this street” (notshown) may be provided. Selection of this link would provide a user withinformation about businesses on the current street, not just on theother side of the street. Still further, a link to video of theneighborhood where the business is located may also be provided. Thevideo provided in response to selection of such a link may be, forexample, a portion of the video data gathered by the vehicle 9 (FIG. 1)as it traveled down the street on which the business identified in thelisting detail page 800 is located. In another embodiment, the detailpage 800 may include a link of “Map this location” (not shown).Selection of this link provides a map view of the current geographiclocation, as described in more detail with respect to FIG. 15.

FIG. 9A illustrates a detailed view of the address information andcorresponding representative image that may be presented in FIG. 7 aspart of the listing 707 or as an alternative to the listing detail page800. The Web page 70 displays the listing 72 of “ACME Pizza,” whichmeets the user's search criteria. The listing 72 includes the name ofthe business, the address and phone number of the business.Additionally, an image 74 of the business previously taken, correlatedwith the address in the listing (i.e., an image of the actual structurecontaining “ACME Pizza” 75 in this case), and identified as therepresentative image (either default or user specific) is displayed.While the image 74 is automatically displayed together with the displayof the listing 72 in this example, it is possible to first display onlythe listing 72 with a “view image” icon (not shown), and then displaythe image 74 only upon the user's selection (e.g., clicking) of the“view image” icon. The Web page 70 of address information andcorresponding image may further display a “zoom in” icon 76, a “zoomout” icon 78, a “scroll to the left” icon 80, a “scroll to the right”icon 82, and a “what's on the other side of the street” icon 84, any ofwhich can be selected by the user to request a specific operationthrough the browser application 69 of the client system 61, as will bedescribed with specific examples below. It should be understood that theindicia of these icons are not limited to those illustrated in FIG. 9A.For example, the “zoom in” and “zoom out” icons may display the actualtext “zoom in” and “zoom out,” as opposed to the plus and minus signsshown in FIG. 9A.

FIG. 9B illustrates a sample Web page 84 created in response to theuser's selection of the “zoom out” icon 78 in FIG. 9A, showing azoomed-out image 85 of “ACME Pizza” 75. Specifically, a wider range ofstreet addresses surrounding (perhaps centering on) the address of “ACMEPizza” is specified according to a predefined program instruction (bythe Yellow Pages server 63 of FIG. 6, for example), and the image (orimages) corresponding to the specified range of street addresses isretrieved and displayed. An arrow 86 may be displayed to indicate where“ACME Pizza” 75 is in this zoomed-out view. If the image 85 cannot befurther zoomed-out, the zoom-out icon 78 need not be displayed and onlythe zoom-in icon 76 may be displayed at this time. The zoomed-out image85 is helpful for the user to appreciate the overall neighborhood viewsurrounding “ACME Pizza.” Referring to a box 87, the neighborhoodinformation, for example, the range of street numbers included in theimage 85, may be displayed.

FIG. 9C illustrates a sample Web page 88 created in response to theuser's selection of the “scroll to the right” icon 82 in FIG. 9B,showing an image 89 of the neighborhood still including “ACME Pizza” 75but as shifted to the right (thus showing “ACME Pizza” 75 at the left ofthe screen). The range of street addresses shown previously in FIG. 9Bmay be shifted to the right by a specific amount according to apredefined program instruction, and the image (or images) correspondingto the shifted range of street addresses is retrieved and displayed. Thearrow 86 may continue to be displayed to indicate where “ACME Pizza” 75is in this scrolled-to-the-right view. The scroll-to-the-right featuremay be limited so that the business of initial interest, “ACME Pizza” 75in this example, will not disappear from the view. Alternatively, thescroll-to-the-right feature (or the scroll-to-the-left feature) may beunlimited and may continue to function as long as the addresses (and theimages) are available to the right or left of the image currently shown.

Additionally, a second arrow 90 may be displayed to point to the address(business) at the center of the displayed image 89 (“Bob's Barbershop”105 in the illustrated example). In this embodiment, the listinginformation (the business name, address, phone number, business hours,etc.) of the business that the second arrow 90 points to may bedisplayed, as in box 91. Thus, as the user scrolls the view to the right(or to the left), the listing information of a different business (ascentered in the image) is sequentially displayed so that the user canappreciate what types of businesses are located along the street. Also,as the user scrolls the view to the right (or to the left), the streetnumber range displayed in box 92 is updated to reflect what numbers areshown in the image at any given time. For example, FIG. 9B showed animage of “ABC Street 1000-1100” (box 87), while FIG. 9C shows an imageof “ABC Street 1050-1150” (box 92), indicating that the image in FIG. 9Chas been shifted by 50 street numbers to the right from FIG. 9B.

FIG. 9D illustrates a sample Web page 93 created in response to theuser's selection of the “what's on the other side of the street” icon 84in FIG. 9C, showing an image 94 of the neighborhood (at the same scale)that is on the other side of the same street as shown in FIG. 9C.Specifically, a range of street addresses that are on the other side ofthe same street as shown in FIG. 9C is identified according to apredefined program instruction, and the image (or images) correspondingto the new range of street addresses is retrieved and displayed. Thesecond arrow 90 points to a business located at the center of the image(i.e., directly across the street from “Bob's Barbershop” 105 in FIG.9C), which is “Carl's Flower Shop” 106, as listed in box 95. As before,the range of street numbers included in the image may be displayed, asin box 96. In the illustrated example, FIG. 9C showed an image of “ABCStreet 1050-1150” (even numbers, in box 92), while FIG. 9D shows animage of “ABC Street 1051-1151” (odd numbers, in box 96). The “what's onthe other side of the street” feature permits the user to appreciate theneighborhood view along either side of the same street. As in FIG. 9C,when the user scrolls the image 94 of FIG. 9D to the right or to theleft, the listing information in the box 95 can be changed to refer tothe business presently centered in the image 94 (and pointed to by thesecond arrow 90).

In FIG. 9D, note that a street 97 extends perpendicular to “ABC Street”(i.e., into the plane of FIG. 9D), indicating the existence of anintersection. Especially at an intersection, the user may wish to obtaina 360-degree view to appreciate the overall atmosphere of the location.Thus, in addition to the images of individual businesses as describedabove, 360-degree views of selected locations such as intersections maybe obtained and correlated with their addresses (e.g., ABC Street andXYZ Boulevard), and stored in the database server 66.

FIG. 9E illustrates a sample Web page 98 suitable for permitting theuser to manipulate the view, for example, to obtain a 360-degree view ofa selected location. Specifically, when a 360-degree view is available,a “Click and Drag” icon 100 may appear to prompt the user to click anddrag a cursor on the image 99 itself in the direction in which the userwishes to see a further view. For example, if the user clicks and dragsa cursor on the image 99 continuously to the right, then the point ofview continuously moves to the right until a complete 360-degree view isobtained. Additionally, clicking and dragging a cursor along a streetmay shift the point of view along the street to obtain a view as if theuser is walking along the street. (Similar to using the “scroll to theright” or “scroll to the left” features, described above.) Specificimplementations of a click-and-drag feature are known in the art ofvirtual reality computing, and need not be described in detail herein.To assist the user in orienting where the user is, a map 101 surroundingthe user's current cursor location 102 on the image 99 may be displayed,together with the direction in which the user's view is facing (in theform of an arrow 103). Still alternatively, it is contemplated that theuser may click on the map 101 to request and display a neighborhood viewof the location clicked, depending on images available in the databaseserver 66.

It is emphasized that FIGS. 9A-9E, as described above, are merelysamples of individual business listings that may be displayedindividually or in conjunction with other businesses in a listing 707(FIG. 7). Additionally, FIGS. 9A-9E illustrate examples of how thosebusiness listings may be manipulated. As discussed above, selection ofthe interactive business name or representative image may result in alisting detail page 800 (FIG. 8) that provides still further informationregarding a business listing.

While it has been described that address-correlated images are initiallystored and retrieved based on a user's inquiry, it is possible to storeimages in association with geographic locations only, and then correlatean address entered or requested by a user with a particular image later,for example, on-the-fly at the time the user specifies an address.

FIG. 10 is a flow diagram of a search results routine utilized toprovide listing information in response to a user-provided searchrequest, in accordance with an embodiment of the present invention. Thesearch routine 1000 begins at block 1001. At block 1003, a search termis received. For example, as discussed above, a user may provide anytype of search term for searching for businesses or other addresses,such as to search for all business of the type “pizza.”

Upon receipt of a search term at block 1003, at block 1005, the locationand search range are determined. The location may be a user-specifiedlocation for which the search is to be performed or determined basedupon user preferences, such as the user's home address. The search rangemay also be specified by the user and may be any predetermined range forwhich a search is to be performed. Once the location and search rangehave been determined, at block 1007, a map, such as a street map, of theidentified location is displayed in a graphical user interface, such asa Web page. In addition to displaying a map of the location at block1007, at block 1009, a search routine 1000 identifies a listingcontained in the listings database 67 that matches the received searchterms. At decision block 1011, it is determined whether a user haspreviously selected a representative image for the identified listing.As discussed above, a representative image may be specified by a userselecting one of a plurality of correlated images for a specific listingas the appropriate representative image for the listing. Any suchselection is stored for that particular user as being that user'sspecified representative image for that address. If it is determined atdecision block 1011 that there is a user-specified representative image,at block 1013 that user-specified representative image is selected.However, if it is determined at decision block 1011 that there is nouser-specified representative image, at block 1015, the defaultrepresentative image assigned to the identified listing is selected. Adefault representative image may be assigned based on either a pluralityof votes from other users selecting a representative image for aparticular listing, based on the geographic coordinates of the addressitself and the correlated image with the closest matching geographiccoordinates, or based on selection by the business owner, to list a fewexamples.

Upon selection of a representative image at either block 1013 or block1015, at block 1017, a position identifier is associated with thelisting. A position identifier may be the number associated with alisting as it is presented to the user or any other identifying criteriasuch as the representative image or an aerial image of the structure atthat address. At block 1019, the identified address information,position identifier, and selected representative image are eachdisplayed to the user via a graphical user interface, such as a Webpage. The address information and corresponding image may be displayedadjacent to one another as a listing, and the position identifier may bepresented in the map displayed at block 1007 at a position where thebusiness is located within the map. For example, referring back to FIG.7, address information and corresponding images, such as addressinformation 713, 723 and corresponding image 727, are displayed adjacentto each other as part of the listing 707. Likewise, a positionidentifier 715 for that address is presented in the map 709.

At decision block 1021, it is determined whether there are anyadditional listings that match the search terms received at block 1003that are within the range identified at block 1005. If it is determinedat decision block 1021 that there are additional matches, the routine1000 returns to block 1009 and continues. However, if it is determinedat decision block 1021 that there are no additional matches, the routinecompletes, as illustrated by block 1023.

FIG. 11 is a flow diagram of a details information routine forpresenting a detailed information view of a selected address, inaccordance with an embodiment of the present invention. The addressselection routine 1100 begins at block 1101. At block 1103, a selectionof an address is received. An address selection may be received inresponse to a user selecting a particular address from a listing view ofseveral addresses. Alternatively, an address selection may be receivedin response to a user providing a search term that only returns onematching search result or by a user providing a particular address forwhich information is desired.

Upon receipt of an address at block 1103, at block 1105, informationassociated with the address, such as business information, and imagespreviously correlated with the particular address are obtained from thelistings database 67 and the images database 68. At decision block 1107,it is determined whether a user has previously selected a representativeimage for the particular address for which a selection was received. Ifit is determined at decision block 1107 that there is a user-specifiedrepresentative image, at block 1109, that user-specified representativeimage is selected for the particular address. However, if it isdetermined at decision block 1107 that there is no user-specifiedrepresentative image, at block 1111, a default representative image isselected. At block 1113, the selected representative image, associatedinformation, and other correlated images for the particular address aredisplayed in a single graphical user interface, such as a Web page. Forexample, referring back to FIG. 8, a representative image 807,associated information 801, 803, 805, and other correlated images 817,819, 821, 823, 825, 827 are displayed in a listing detail page 800.

As discussed above, defining boundaries for an address and correlatingall images geographically-located within a defined boundary may resultin multiple images being associated with the same address. Thosecorrelated images may be displayed based on their geographic location,the time at which they were taken, based on the season, etc., therebyresulting in what appears to be an overall filmstrip-view of the blockon which the selected address is located. The filmstrip-view of imagesmay include the selected representative image that is also displayed asa larger image in the same graphical user interface.

In addition to displaying the selected representative image, associatedinformation, and other correlated images, at block 1115, links toaddresses near the selected address may also be provided. The links tonearby addresses may be interactive, and selection of one of those linksmay result in the routine 1100 being performed for the addressidentified by the selected link and ultimately a new graphical userinterface being provided, containing information for the selectedaddress, a representative image and other correlated images.

At decision block 1119, it is determined whether the selected addressreceived at decision block 1103 is a business address. Such adetermination may be made by reviewing the address contained in thelistings database 67. If it is determined that the selected address isnot a business address, the address selection routine 1100 may proceedto the voting subroutine illustrated by block 1127 and described in moredetail with respect to FIG. 12. In an alternative embodiment, if it isdetermined that the selected address is not a business address, e.g., itis a residential address, the routine 1100 may obtain and provideadditional information, such as a telephone number for the address.However, as illustrated in FIG. 11, providing additional information fora non-business address may not be performed due to individual privacyconcerns.

However, if it is determined that the selected address is a businessaddress, at block 1121, the type of business address is determined byobtaining such information from the listings database 67. Upondetermination of a business type, at block 1123, information aboutsimilar businesses and links to those businesses may be provided. Forexample, a single link to similar businesses may be provided in a detailview, such as the listing detail page 800 (FIG. 8). Selection of a linkto similar businesses may result in a new listings display window (suchas listings page 700 of FIG. 7) identifying one or more businessaddresses and corresponding representative images. Alternatively,several links may be included in the graphical user interfaceidentifying several businesses similar to the business type of thebusiness located at the selected address received at block 1103. Atblock 1125, a determination is made as to whether a user has provided aselection of what they view as the appropriate representative image. Ifit is determined at block 1125 that a user has provided a selection of arepresentative image, the address selection routine 1100 continues tothe voting subroutine, as illustrated by subroutine block 1127 anddescribed in detail below. However, if it is determined at decisionblock 1125 that a user has not selected a new representative image orupon completion of the voting subroutine 1127, the address selectionroutine 1100 completes, as illustrated by block 1129.

FIG. 12 is a flow diagram of a voting subroutine for designating arepresentative image, for a geographic location in response to userselections, in accordance with an embodiment of the present invention.The voting subroutine 1200 begins at block 1201. At block 1203, a votefor a best image is received from a user. As discussed above, in adetails graphical user interface, such as listing detail page 800 (FIG.8), a user may be presented with a plurality of images, each of which iscorrelated to a particular address. A user may select one of thoseimages as being the one that the user perceives as the appropriaterepresentative image for the corresponding address. Upon receipt of aselection or vote for a representative image at block 1203, at block1205, the vote is stored and associated with the image and address.Additionally, at block 1207, the selected image and the address areassociated with the voter (user) so that future selections of thataddress by that user result in the selected image being displayed as therepresentative image for that address. At decision block 1209, adetermination is made as to whether the voted upon image has a majorityof votes for the particular address. If it is determined at decisionblock 1209 that the voted image does not have a majority of the votes,the subroutine completes, as illustrated by block 1213. However, if itis determined that the voted image does have a majority of the votes forthat particular address, at block 1211, that image is identified asbeing the default representative image for that address. As discussedabove, the default representative image is provided to users who havenot previously selected a representative image for that address.

FIG. 13 is a flow diagram of an image correlation correction routine foradjusting the address coordinates for the geographic location of aparticular address and re-correlating images to a particular address,according to an embodiment of the present invention. The imagecorrelation correction routine 1300 begins at block 1301. At block 1303,a user voted upon default representative image for a particular addressis identified. As described above, a default representative image for aparticular address may be identified based on a plurality of votesreceived from different users voting as to which image is perceived tobe the appropriate representative image for a particular address. Uponidentification of a default representative image for an address at block1303, at decision block 1305, it is determined whether the coordinatesfor the image are sufficiently different from the coordinates of theaddress. The coordinate comparison at decision block 1305 may bedetermined based on a predetermined threshold. For example, if it isdetermined that the difference between the address coordinates and thedefault representative image coordinates is beyond a predeterminedthreshold, at block 1307, the address coordinates are adjusted to matchthe default representative image coordinates. However, if it isdetermined at decision block 1305 that the coordinates are notsufficiently different, the routine completes at block 1313.

Upon address coordinate adjustment at block 1307, at block 1309, anaddress boundary subroutine is performed, as described in more detailbelow with respect to FIG. 14. Upon completion of the address boundarysubroutine at block 1309, at block 1311, images from the images database68 are re-correlated based on the new address boundary, or boundaries,defined by the address boundary subroutine 1400. Upon re-correlation ofthe images at block 1311, or if it is determined at decision block 1305that the coordinates of the address and the default representative imageare not sufficiently different, the image correlation correction routine1300 completes at block 1313.

FIG. 14 is a flow diagram of an address boundary definition subroutinefor defining address boundaries, in accordance with an embodiment of thepresent invention. The address boundary subroutine 1400 begins at block1401. At block 1403, address coordinates for an address are received. Asdiscussed above, address coordinates may be obtained from a listingsdatabase 67 or may be adjusted address coordinates generated as a resultof the image correlation correction routine 1300 (FIG. 13). Upon receiptof address coordinates, at block 1405, addresses on the same side of thestreet as the received address are identified. At block 1407, the typeof object at each address is identified. For example, as discussed abovewith respect to FIG. 5, it may be identified that one of the addressesis for a grocery store, another a shoe repair store, and another aconvenience store.

At block 1409 it is determined which of the identified addresses areadjusted coordinate addresses, and for each adjusted coordinate addressat block 1409, an image boundary is defined based on the type of objectat the address. At block 1411, for the remaining addresses, i.e., theaddresses that are not adjusted coordinate addresses, the imageboundaries are defined adjacent to the defined address boundariesdefined at block 1409. The image boundaries defined at block 1411 aredefined and positioned based on the type of object located at theaddress and based on the boundaries defined in block 1409. For example,referring back to FIG. 5, if the addresses 505 and 511 are adjustedaddresses, i.e., the coordinates for the addresses have been adjusted tocoincide with the coordinates of user selected default representativeimages, the boundaries 507, 511 are defined with respect to the type ofobject and the address coordinates. Subsequently, the boundary foraddress 503 would be defined based on the type of object located ataddress 503 and based on the defined boundaries 507, 511. In particular,the boundary for address 503 would be defined such that it encompassesthe area between boundaries 507 and 511, as it is highly likely that theimages between boundaries 507 and 511 are representative of the address509. Additionally, the boundary 509 may be defined such that it eitheroverlaps the adjacent boundaries, as illustrated by overlap 513, or doesnot overlap adjacent boundaries. The address boundary subroutine 1400completes at block 1413.

In addition to generating online directories, address-correlated images,or images with geographic information associated therewith, as describedabove, embodiments of the present invention may also be used to generatemap views of geographic areas. FIG. 25 illustrates one environment forimplementing an online mapping interface that includes a mapping system2560 and one or more client systems 2561 (only one is shown). In anillustrative embodiment, the client systems 2561 may include personalcomputers, hand-held computers, mobile telephones, PDAs, workstations,and the like, that individual users can use to communicate with themapping system 2560. In the environment shown in FIG. 25, the clientsystems 2561 communicate with the mapping system 2560 via wired orwireless connections to one or more communication networks, such as theInternet 2562. Protocols and components for communicating via theInternet are well known to those of ordinary skill in the art ofcomputer network communications. The client system 2561 requests andretrieves information from the mapping system 2560 via a Web browserapplication 2569 that provides the user with an interface to theInternet, as is also well known to those of ordinary skill in the art.

The mapping system 2560 depicted in FIG. 25 operates in a distributedcomputing environment comprising several computer systems that areinterconnected via communication links, e.g., using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that the system 2560 could equally operatein a computer system having fewer or greater number of components thanare illustrated in FIG. 25. Thus, the description of the mapping system2560 in FIG. 25 should be taken as exemplary and not limiting to thescope of the invention. In particular, the mapping system 2560 maycomprise plural subsystems, each for performing a specific function inresponse to a request from a client system.

The mapping system 2560 as illustrated includes both a mapping server2563 having an operating system 2564 and a search engine 2565, and adatabase server 2566 having a listings database 2567, an images database2568, and optionally a map database 2570. The mapping server 2563, inthis example, is a server that generates content corresponding to mapsof geographic areas that can be displayed together with imagescorrelated with specific locations included within the displayed map. Inan illustrative embodiment, the mapping server 2563 communicates textand graphics organized as a Web page using, for example, hypertexttransfer protocols. The mapping server 2563 also conducts searches ofthe listings databases 2567, images database 2568, and the map database2570 in the database server 2566. The administration and overalloperation of the mapping server 2563 is controlled by its operatingsystem 2564, while its search function is performed by the search engine2565 in cooperation with the database server 2566.

The database server 2566 maintains the listings database 2567 and theimages database 2568 that the mapping server 2563 uses to respond touser requests and search queries. The listings database 2567 containsinformation generally found in conventional Yellow Pages in a searchableform, i.e., numerous listings (business names together with theiraddresses, phone numbers, business type, etc.) that are indexed andsearchable by various search criteria. The listings database 2567 mayalso contain information such as the geographic coordinates of thebusiness location, the business hours, a brief description or profile,customer reviews, etc., of each business listed. The images database2568 contains digitized images of the businesses in correlation with thebusiness addresses (i.e., street addresses) and/or geographiccoordinates stored in the listings database 2567. The map database 2570includes digitized maps of geographic regions that have correspondingimages stored in the images database 2568. In an alternative embodiment,the map database 2570 may be omitted and street maps may be obtainedfrom third party map providers, such as MapQuest®, and used withembodiments of the present invention.

The database server 2566 is configured to receive search queries fromthe search engine 2565 and return search results from the listingsdatabase 2567, the map database 2570, and the images database 2568.Those of ordinary skill in the art will recognize that the mappingserver 2563 and the database server 2566 will each typically employvarious computing components for storing and executing programinstructions for operation of the servers.

The mapping system 2560 enables a user to search and view maps (e.g.,street maps) of geographic locations in which some or all of thestructures surrounding the streets on the map can be representedgraphically using an image (e.g., business building) taken at thegeographic location of the business that is correlated with the addressof the geographic location. FIGS. 15-23 illustrate sample screendisplays with graphical user interfaces that can be displayed on theclient system 2561 in response to user requests and search queries, inaccordance with embodiments of the present invention.

FIG. 15 illustrates a map page 1500 returned to a client system 2561when a user requests to view information on a location, in accordancewith an embodiment of the present invention. In response to a userproviding a location in the location box 1525 and selecting the mapbutton 1526, the mapping system 2560 returns a street map 1501 centeredon the location entered in the location box 1525. The entered locationmay be a general location, such as “Seattle, Wash.,” or a specificlocation, such as “1000 Boren Ave., Seattle, Wash.” The position of thespecified location in the street map 1501 is identified by a positionidentifier 1509. The position identifier 1509 may be any type ofgraphical icon or image, including, but not limited to, a magnifyingglass, a star, a car, etc. The position identifier 1509 may also becustomizable or vary according to user preferences and/or detectedenvironment.

The mapping system 2560 may also provide in the map page 1500 arepresentative image 1505 of the search location and secondary images1507A-1507H surrounding the search location. As discussed below, theposition identifier 1509, the representative image 1505, and thesecondary images 1507A-1507H are interrelated and interaction with anyone of the position identifier 1509, representative image 1505, orsecondary images 1507A-1507H may result in a change to the others.

The position identifier 1509 may include scroll icons 1511, 1512 thatmay be selected by a user to move the position identifier 1509 in aparticular direction along a street displayed on the street map 1501.The scroll icons 1511, 1512 are oriented around the position identifierso that they are parallel with the street on which the positionidentifier 1509 is located. For example, if the position identifier ispositioned on a street that runs north and south, the scroll icons willpoint north and south, parallel with the direction of the street.Selection of the scroll icon that is pointing north will result in theposition identifier moving in the northward direction along the street.The representative image 1505 and secondary images 1507A-1507H willautomatically be adjusted to display a representative image andsecondary images of the new location of the position identifier. Thescroll icons 1511, 1512 may be displayed in accordance with a known orperceived path of motion. Additional scroll icons (not shown) may alsobe displayed if a user has additional directions of travel (e.g., if thespecified location corresponds to an intersection).

In the example illustrated in FIG. 15, the images associated with thelocation “1000 Boren Ave., Seattle, Wash.” include a representativeimage 1505 of that location and secondary images 1507A-1507H that aregeographically near that location. If the searched for location isgeneral in nature, such as “Seattle, Wash.,” the associated images maybe images at the center of the search location, or images commonlyassociated with the general location. For example, the representativeimage for the general location “Washington, D.C.” may be an image of theWhite House. The secondary images may be identified as those imagessurrounding the location of the selected representative image. Therepresentative image 1505 and secondary images 1507A-1507H for aspecific location may be selected based on geographic coordinates or byidentifying images that are correlated with the specific location.

The secondary images that are nearby or correlated with the searched forlocation may be presented in a single or dual filmstrip-type view 1508.The images within the dual filmstrip-type view 1508 may be furtherorganized according to geographic location surrounding a street on whichthe search location is geographically positioned. For example, thesearch location “1000 Boren Ave., Seattle, Wash.” has a representativeimage 1505 and a filmstrip-type view 1508 of secondary images 1507A,1507B, 1507C, and 1507D that are on the same side of the streetrepresentation 1507 “Boren Ave” as the representative image 1505. As canbe seen, the secondary image 1507C is also the representative image1505. Secondary images 1507E, 1507F, 1507G, and 1507H are displayed in afilmstrip-type view 1508 on the opposite side of the streetrepresentation 1507. As will be appreciated, the dual filmstrip-typeview may represent any collection of secondary images, and the dualfilmstrip-type view of images geographically organized on opposite sidesof a street is provided only as an example. For example, the dualfilmstrip-type view of secondary images may represent only images on oneside of the street, a montage of images surrounding the search locationor representative of the search location, a panoramic view of the searchlocation, a collection of images representative of historical markersnear the search location, etc. Still further, a user may select theorganization type for the dual filmstrip-type view of secondary images.

Also included in the map page 1500 are scroll icons 1513, 1515, 1517,1519. Similar to interaction with the scroll icons 1511, 1512,interaction with any of the scroll icons 1513, 1515, 1517, 1519 resultsin the position identifier 1509, the interrelated representative image1505, and the secondary images 1507A-1507H being adjusted. In anillustrative embodiment, the scroll icons 1513, 1515, 1517, 1519 can bedirectionally correlated with the orientation of the scroll icons 1511,1512. For example, scroll icons 1515, 1517 can be directionallycorrelated to scroll icon 1511 such that interaction with one willresult in the retrieval of new images in the direction of orientation ofscroll icon 1511. In one embodiment, the scroll icons 1511, 1512, 1513,1515, 1517, 1519 may be graphically distinguishable to identify variousrelationships among the icons and/or the degree to which the otherdisplayed content will be manipulated. For example, scroll icons 1511,1515, 1517 may be one color, in this example black, to identify theirrespective relationships and scroll icons 1512, 1513, 1519 may be adifferent color, in this example white, to illustrate their respectiverelationships. Interaction with one of the black scroll icons 1511,1515, 1517 results in a change to the position identifier 1509, therepresentative image 1505, and the secondary images 1507A-1507H in thedirection illustrated by the other black scroll icons. Likewise,interaction with one of the white scroll icons 1512, 1513, 1519 resultsin a change to the position identifier 1509, the representative image1505, and the secondary images 1507A-1507H in the direction illustratedby the other white scroll icons. In another example, the scroll iconscan have different sizes and/or ranges to graphically indicate how muchadjustment will be made to the corresponding images and icons.

While the embodiment described herein utilizes black and white tographically distinguish the scroll icons, any type of graphicalrepresentation may be used. For example, each set of related scrollicons may be displayed using hatching or different colors. Optionally,or in addition thereto, the related sets of scroll icons may each bedisplayed using different shapes.

Selection of either walk icon 1521 or walk icon 1523 results in adifferent image, such as the image geographically adjacent to thecurrent representative image, being presented as the representativeimage 1505. In this example, if a user selects walk icon 1523, thecorresponding image next to the current representative image 1505(1507C), in this example secondary image 1507D, will be displayed as therepresentative image 1505. Additionally, if the current representativeimage is at the end of the filmstrip-type view, the filmstrip-type views1508 of the secondary images will move in either direction and display anew set of secondary images, as if the user is walking down the street.

The scroll icons 1513, 1515, 1517, 1519 displayed at either end of thedual filmstrip-type views 1508 of the secondary images allow a user toview the next set of secondary images on either side of the street. If auser selects the black scroll icon 1517, a new set of secondary images,in this case four new images illustrated in FIG. 16, are displayed inthe filmstrip-type view 1608 above the street representation 1607.Likewise, the set of secondary images on the other side of the street inthe direction of the related black scroll icon 1615 will be adjustedsuch that the new set of secondary images appearing below the streetrepresentation 1607 are the secondary images 1607E-1607H directly acrossthe street from the currently displayed secondary images 1607A-1607Dabove the street. The representative image 1605 is also adjusted todisplay a new representative image from the current set of secondaryimages 1607A-H. The position identifier 1609 is also moved in thedirection of the related black scroll icon 1611 to identify the locationon the street map 1601 of the currently displayed representative image1605 and secondary images 1607A-1607H.

A user may also select the position identifier 1609 and reposition it toa new location within the street map 1601. In response, the mappingsystem 2560 will identify the images associated with the newly selectedlocation on the street map 1601 and display those images as therepresentative image and correlated images. Referring to FIG. 17, Inresponse to a user moving the position identifier 1709 to a new locationwithin the street map 1701, the mapping system 2563 identifies arepresentative image 1705 correlated with the new location of theposition identifier 1709 and secondary images 1707A-1707H surroundingthe new location. Additionally, the street representation 1707 displayedbetween the two filmstrip-type views 1708 of the secondary images1707A-1707H is updated to include the name of the street on which theposition identifier 1709 is now located.

Through interaction with the map page 1700, a user may change the imagethat is presented as the representative image 1705 and change thecorrelated images 1707A-1707H. Similar to the secondary images displayedin the listing detail page 800 (FIG. 8) and discussed above, thesecondary images 1707A-1707H may be interactive in that a user mayposition a pointing device, such as a mouse pointer, over one of theimages and that image will temporarily replace the representative image1705. Referring to FIG. 18, if a user manipulates a pointing device oversecondary image 1807G (such as by hovering over the image), therepresentative image 1805 is replaced using the image 1807G as the newrepresentative image. Additionally, a user may also select therepresentative image 1805 or one of the secondary images 1807A-1807H andbe provided with a closer view (not shown) of the image selected.Alternatively, selection of a representative image 1805 or a secondaryimage 1807A-1807H may generate a listing details page 800 (FIG. 8) forthe location associated with the selected image.

A user may also interact with the map page 1800 to adjust the zoom sizeor position of the street map 1801 through interaction with the slidingscale 1833 or the compass 1831. The sliding scale 1833 allows a user toadjust the amount of geographic area represented in the street map 1801.If the user moves the sliding scale 1833 up, the amount of geographicarea represented in the street map 1801 decreases (i.e., the street map1801 is zoomed-in). Likewise, if the user moves the sliding scale 1833down, the amount of geographic area represented in the street map 1801increases (i.e., the street map 1801 is zoomed-out). Interaction withthe compass 1831 results in the street map 1801 scrolling to the nextgeographic area in the direction of the interaction with the compass1831. For example, if a user selects the “West” arrow on the compass1831, the street map 1801 will scroll to display the geographic area tothe west of the currently displayed geographic area.

The map page 1800 may also include a map location window 1803 showing azoomed-out view of the geographic area that the street map 1801 isdisplaying. In one example, the street map 1801 may be represented inthe location map 1803 by a box 1834. Likewise, the position identifier1809 may be reproduced in the location map 1803 and represented by asecond position identifier 1832. A user may interact with the secondposition identifier 1832, or box 1834, in the location map 1803 toreposition the location for which images and the street map 1801 areprovided. For example, a user may select the position identifier 1832 orbox 1834 and reposition it to a different location on the location map1803. In response, the street map 1801 is updated to display the newlocation in a location window 1803. Additionally, the mapping system2560 locates the images correlated with the new location and displaysthe representative image 1807 and the secondary images 1807A-1807H inthe map page 1800.

FIG. 19 is a map page 1900 identifying locations for which images havebeen captured, in accordance with an embodiment of the presentinvention. Through selection of the “Mark Streets Containing Images”1929, the street map 1901 includes a graphical representation 1935identifying which streets or locations have correlated images. Forexample, the streets covered with hatched lines 1935 illustrate streetswith correlated images that are viewable by a user. In contrast, streetswithout hatched lines, such as “Columbia Street” 1937 and “8th Avenue”1939, do not include correlated images. In one embodiment, if a userselects a street that does not include correlated images, such as“Columbia Street” 1937, no representative and/or secondary images willbe displayed in the map page 1900. In an alternative embodiment, themapping system 2560 may display the representative image and secondaryimages that are geographically closest to the selected location andprovide an identifier (not shown) indicating that the selected locationhas no correlated images but those closest by are displayed. Stillfurther, the mapping system 2560 may display a place holder identifyingthat the images are not available, or “coming soon.”

FIG. 20 illustrates a mapping page allowing a user to enter specificstart and end locations for obtaining driving directions, in accordancewith an embodiment of the present invention. To implement this feature,a user may select the “Driving Directions” link 1527 (FIG. 15) and beprovided with “From” location box 2039 and “To” location box 2041. Usingthe “From” location box 2039 and “To” location box 2041, the user mayinput a starting location and/or an ending location and receivedirection details between the two provided points. If the user haspositioned the position identifier 2009 over the location from whichthey wish to travel, the user may select the From input icon 2024A andthe specific address at the location of the position identifier 2009will be included in the From box 2039. Similarly, if the user has placedthe position identifier 2009 over the location to which they wish totravel, a user may select the To location icon 2024B and the mappingsystem 2560 will include the specific address at the location of theposition identifier 2009 in the To box 2041. Upon selection of the GOicon 2043, direction details between the location in the From box 2039and the location in the To box 2041 will be determined and provided to auser as illustrated in FIG. 21A.

FIG. 21A illustrates a directions page 2100 including detailed mappingdirections (direction details) between two locations, in accordance withan embodiment of the present invention. Continuing with the previousexample, if the user selects to obtain driving directions between twolocations, the mapping system 2560 will determine the specificdirections between those two locations. Upon determining specificdirections between the two locations, the street map 2101 is updated toprovide a graphical representation 2115 illustrating the driving routebetween the two locations. In addition, direction details 2103 areprovided identifying each of the turns and the distance that needs to bemade to drive between the two locations.

In addition to receiving the direction details 2103 and the graphicalrepresentation of the route 2115 illustrating those direction details,the directions page 2100, similar to the map page discussed above,includes a position identifier 2109 with scroll icons 2111, 2112. Aswith above, a user may reposition the position identifier 2109 byselecting and moving the position identifier 2109 or selecting one ofthe scroll icons 2111, 2112. However, in one embodiment, the positionidentifier 2109 may be associated with the direction details 2103 suchthat a user may select one of the scroll icons 2111, 2112 and theposition identifier will move along the route 2115 of the directiondetails. The representative image 2105 and secondary images 2107A-2107Hassociated with the location of the position identifier 2109 may also bedisplayed. Similar to the discussion above, the scroll icons 2111, 2112of the position identifier 2109 are oriented such that they are parallelwith the street on which the position identifier is located. Thus, ifthe user selects the white scroll icon 2112, the position identifier2109 will move in the direction of that icon along the route 2115. Therepresentative image 2105 and secondary images 2107A-2107H for the newlocation will be provided. If the position identifier reaches a turn inthe route, the scroll icons 2111, 2112 will reorient to be parallel withthe street following the turn that is included in the direction details2103.

Additionally, each of the direction details 2103 may be user selectable.In response to a selection of a direction detail 2103, the positionidentifier 2109 will be repositioned to the selected location and arepresentative image and secondary images of that location will beprovided to the user, as illustrated in FIG. 21B. If there are no imagescorrelated with the location of the direction detail, a place holder maybe displayed. For example, if the direction detail is a turn, the placeholder may be an icon indicating that the direction detail is a turn.Additionally, the place holder may indicate the direction of the turnand/or the degree of the turn (hard right, slight right, etc.).Alternatively, if there are no correlated images for the location of thedirection detail, the system may provide an available image along theroute that is closest to the location of the direction detail andoptionally indicate that this image is “nearby” the direction detail.

FIG. 21B illustrates a directions page 2100 identifying a representativeimage 2105 and secondary images 2107A-2107H associated with a particularportion of the driving directions, in accordance with an embodiment ofthe present invention. Continuing with the above example, if a userselects the specific driving direction of “Turn LEFT onto MARION ST.”2110, the position identifier 2109 is repositioned to that location onthe street map 2101. A representative image 2105 of that location isdisplayed to the user along with dual filmstrip-type 2108 views ofsecondary images correlated with or geographically near that location.

FIG. 22 is a flow diagram of a map search results routine for providinga map view of a search location, in accordance with an embodiment of thepresent invention. The map search result routine 2200 begins at block2201 and at block 2203 a search location is received. As discussedabove, a user may provide a specific search location or select alocation on a map. Additionally, if a user is viewing a listing detailpage (FIG. 8), a user may select to map the area represented in thelisting detail page 800. Upon receipt of a search location, at block2205, a range surrounding the location is determined. Determination of asurrounding range may be accomplished using a variety of techniques. Forexample, if the search location is in a densely populated area, thesurrounding range may be small (e.g., two miles), as there are likelyseveral images for that area. In contrast, if the search location is ina rural area, the surrounding range may be much larger to includeadditional images.

Upon determination of a surrounding range at block 2205, at block 2207,images associated with the search location are determined and obtainedfrom the image database. At block 2209, a map, such as a street map,surrounding the search location is displayed to a user in a map page anda position identifier is represented on the displayed map to identifythe search location. Additionally, a representative image associatedwith the search location is displayed and the secondary imagessurrounding and/or correlated with the search location are displayed tothe user. As discussed above, the correlated images may be displayed ina dual filmstrip-type view illustrating images on either side of astreet of the search location, images on the same side of the street asthe search location, a montage of images representative of the searchlocation (or the surrounding area), a panoramic view of the areasurrounding the search location, etc.

At decision block 2213, a determination is made as to whether locationswithin the determined range that have correlated images are to beidentified to the user. If it is determined at decision block 2213 thatlocations within the determined range that have correlated images are tobe identified to a user, at block 2215, the locations within thedetermined range are identified. At block 2217, the locations withcorrelated images are represented on the street map by emphasizing thestreet corresponding to those locations. For example, referring back toFIG. 19, streets containing locations with correlated images may beemphasized by providing a hatching over those streets. However, if it isdetermined at decision block 2213 that locations within the range thathave correlated images are not to be identified to the user, or afteremphasizing those locations at block 2217, the map search result routine2200 completes, as illustrated by block 2219.

FIG. 23 is a map reposition routine for repositioning a positionidentifier and adjusting the displayed representative and correlatedimages, in accordance with an embodiment of the present invention. Themap reposition routine 2300 begins at block 2301 and at block 2302 areposition request is received. At decision block 2303, a determinationis made as to whether the reposition request is for an adjacent image.As discussed above, a request for an adjacent image may result from auser selecting a walk icon. If it is determined at decision block 2303that the reposition request is a request for an adjacent image, at block2305, the position identifier is adjusted on the street map to thelocation of the selected adjacent image and a new representative imageis displayed.

However, if it is determined at decision block 2303 that the request isnot for an adjacent image, a determination is made at decision block2305 as to whether the reposition request is a request for an adjacentset of correlated images. As discussed above, an adjacent set ofcorrelated images may be requested by a user selecting one of the scrollicons located near the secondary images, or by interacting with eitherof the scroll icons of the position identifier displayed on a streetmap. If it is determined at decision block 2305 that the request is notfor an adjacent set of images, at block 2307, the new location of theposition identifier is determined. As discussed above, if a userrepositions the position identifier to a new location, a newrepresentative image and set of secondary images associated with the newlocation will be provided to the user.

If it was determined at decision block 2305 that the reposition requestwas a request for an adjacent set of correlated images, at block 2309,the position identifier displayed on the street map is repositioned at alocation corresponding to the next set of adjacent images. At block2311, the images associated with the location are obtained. At block2313, the representative image is determined and displayed to a user andthe secondary images surrounding that representative image are alsodisplayed to the user. The map reposition routine 2300 completes atblock 2315.

FIG. 24 is a flow diagram of a map directions routine for displayingdirections between two locations along with representative imagesbetween those two locations, in accordance with an embodiment of thepresent invention. The map directions routine 2400 begins at block 2401.At block 2403, a starting location and an ending location are received.As discussed above, a user may input the starting and ending locationsor select those locations from a street map view by interacting with theposition identifier. Upon receiving a starting location and endinglocation at block 2403, at block 2405, a route between those twolocations is determined. As will be appreciated by one of skill in therelevant art, there are many techniques for determining routes betweentwo locations. For example, the shortest physical driving route betweentwo locations may be determined, the route with the shortest calculatedtime based on driving speeds for streets between those two locations maybe determined, a route with the least amount of traffic may bedetermined, etc. Any technique for determining a route between twolocations may be used with embodiments of the present invention.

Upon determination of a route between two locations, a set of directiondetails is generated at block 2407. The direction details may include,but are not limited to, names of streets, direction of turns, drivinglength on each street, and total drive time. At block 2411, thedirection details are displayed to the user identifying the routebetween the two locations. Additionally, on the street map, the streetsthat are to be driven may be emphasized using a bold, hatched, or anyother type of graphical emphasis to display the route that is to betaken between the two locations. At block 2412, the map with the routeemphasized is displayed to the user.

At decision block 2413, a determination is made as to whether imagescorresponding to one of the direction details, or along the route, areto be displayed. A representative image and secondary images may bedisplayed in response to a user selecting one of the driving directiondetails, interacting with the position identifier, or interacting withone of the scroll icons. If it is determined at decision block 2413 thata representative image and secondary images are to be displayed, atblock 2415 a representative image and the secondary images for theselected driving direction detail, or location along the route, areidentified. At block 2417 the identified representative image and thesecondary images are displayed. In an embodiment, a representative imagecorresponding to each direction detail may be displayed next to thedirection detail as a thumbnail image. The displayed representativeimage and the direction detail may be interactive and selection ofeither will provide a user with a larger view of the representativeimage and the secondary images.

If it is determined at decision block 2413 that a representative imageand secondary images are not to be displayed, or after a representativeimage and secondary images have been displayed, at block 2419 the mapdirections routine 2400 completes.

While various embodiments of the invention have been illustrated anddescribed, it will be appreciated that changes can be made thereinwithout departing from the spirit and scope of the invention. The scopeof the invention should be determined from the following claims andequivalents thereto.

What is claimed is:
 1. A system comprising: at least one processor; and memory including instructions that when executed by the at least one processor cause the system to: determine geographical coordinates of a physical location of a computing device; determine a business located at the geographical coordinates based in part on a correlation of a business database and associated location information; provide the computing device with an input option to provide an image of the physical location; receive the image of the physical location; relate the image to one or more of the geographical coordinates and the physical location; and index the image for display when a search associated with the physical location is performed.
 2. The system of claim 1, wherein the instructions when executed by the at least one processor further cause the system to: receive a request to display data for the physical location; display a portion of a map including the physical location and an icon displayed at the location; and display the image as representative of the business or the physical location along with the portion of the map.
 3. The system of claim 1, wherein the instructions when executed by the at least one processor further cause the system to: receive additional images of the physical location; correlate the additional images to the image; determine a quality of each of the correlated images; and provide at least one image from the correlated images as a representative image of the business or the physical location.
 4. The system of claim 1, wherein the instructions when executed by the at least one processor further cause the system to: receive additional images of the physical location at different times of capture; correlate the additional images to seasons associated with the times of capture; receive a search associated with the physical location; determine a current season during which the search is received; and provide at least one image from the additional images as a representative image of the business or the physical location based at least on the current season.
 5. The system of claim 1, wherein the instructions when executed by the at least one processor further cause the system to: allow the computing device to provide a selection from additional images of the physical location separated from the image of the physical location; and index the image for display along with the additional images based at least in part on the selection when a search of the physical location is performed.
 6. The system of claim 1, wherein the instructions when executed by the at least one processor further cause the system to: receive, in the computing device, a change to be applied to an address associated with the physical location; and index the image with the applied change to the address.
 7. A computer-implemented method, comprising: determining geographical coordinates of a physical location of a computing device; determining a business located at the geographical coordinates based in part on a correlation of a business database and associated location information; receiving an image of the physical location based in part on an input from the computing device; relating the image to one or more of the geographical coordinates and the physical location; and indexing the image for display when a search associated with the physical location is performed.
 8. The computer-implemented method of claim 7, further comprising: receiving a request to display data for the physical location; displaying a portion of a map including the physical location and an icon displayed at the location; and displaying the image as representative of the business or the physical location along with the portion of the map.
 9. The computer-implemented method of claim 7, further comprising: receiving additional images of the physical location; correlating the additional images to the image; determining a quality of each of the additional images; and providing at least one image from the correlated images as a representative image of the business or the physical location.
 10. The computer-implemented method of claim 7, further comprising: receiving additional images of the physical location at different times of capture; correlating the additional images to seasons associated with the times of capture; receiving a search associated with the physical location; determining a current season during which the search is received; and providing at least one image from the additional images as a representative image of the business or the physical location based at least on the current season.
 11. The computer-implemented method of claim 7, further comprising: allowing the computing device to provide a selection from additional images of the physical location separate from the image of the physical location; and indexing the image for display along with the additional images based at least in part on the selection when a search of the physical location is performed.
 12. The computer-implemented method of claim 7, further comprising: receiving, in the computing device a change to be applied to an address associated with the physical location; and indexing the image with the applied change to the address.
 13. The computer-implemented method of claim 7, further comprising: displaying a graphical element indicating a detected direction of orientation of the computing device with respect to a reference coordinate system; and indexing the orientation of the computing device with the image.
 14. The computer-implemented method of claim 13, further comprising: receiving a request to display data for the physical location; displaying a portion of a map including the physical location and an icon displayed at the location; displaying the image as representative of the business or the physical location along with the portion of the map; and when the image is selected, updating an orientation of the portion of the map to match the detected direction of orientation with respect to the reference coordinate system.
 15. The computer-implemented method of claim 13, further comprising: receiving a request to display data for the physical location; displaying a portion of a map including the physical location and an icon displayed at the location; displaying the image as representative of the business or the physical location along with the portion of the map; and when the map is scrolled, replacing the image with a new image to match a new physical location in the detected direction of the scrolling.
 16. The computer-implemented method of claim 7, further comprising: receiving a selection to view data for the physical location; and displaying a three-hundred and sixty degree view that includes the image.
 17. A non-transitory computer readable storage medium storing instructions executable by at least one processor to cause the at least one processor to perform operations comprising: determining geographical coordinates of a physical location of a computing device; determining a business located at the geographical coordinates based in part on a correlation of a business database and associated location information; receiving an image of the physical location based in part on an input from the computing device; relating the image to one or more of the geographical coordinates and the physical location; and indexing the image for display when a search associated with the physical location is performed.
 18. The non-transitory computer readable storage medium of claim 17, further comprising instructions executable by the at least one processor to cause the at least one processor to perform further operations comprising: receiving a request to display data for the physical location; displaying a portion of a map including the physical location and an icon displayed at the location; and displaying the image as representative of the business or the physical location along with the portion of the map.
 19. The non-transitory computer readable storage medium of claim 17, further comprising instructions executable by the at least one processor to cause the at least one processor to perform further operations comprising: stitching together at least a portion of additional images determined to be geographically adjacent; providing a three-hundred and sixty degree view of a region surrounding the physical location, wherein the region is centered proximate to the physical location.
 20. The non-transitory computer readable storage medium of claim 17, further comprising instructions executable by the at least one processor to cause the at least one processor to perform further operations comprising: allowing the computing device to provide a selection from additional images than the image of the physical location; and indexing the image for display along with the additional images based at least in part on the selection when a search of the physical location is performed. 